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METHOD AND SYSTEM FOR REDUCING THE FALSE ALARM RATE 
OF NETWORK INTRUSION DETECTION SYSTEMS 

TECHNICAL FIELD OF THE INVENTION 

This invention relates generally to intrusion 
detection and, more particularly, to a method and system 
for reducing the false alarm rate of network intrusion 
detection systems using offline passive analysis. 
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BACKGROUND OF THE INVENTION 

Network Intrusion Detection Systems ("NIDS") are 
typically designed to monitor network activity in real- 
time to spot suspicious or known malicious activity and 
5 to report these findings to the appropriate personnel. 
By keeping watch on all activity, NIDS have the potential 
to warn about computer intrusions relatively quickly and 
allow administrators time to protect or contain 
intrusions, or allow the NIDS to react and stop the 
10 attack automatically. In the security industry, a NIDS 
may either be a passive observer of the traffic or an 
active network component that reacts to block attacks in 
real-time . 

False alarms in an NIDS may be reduced by using a 
15 technique called passive operating system (OS) analysis. 
The typical implementation watches network traffic in 
real-time to discern the operating system types of the 
hosts by looking at the raw network packets and matching 
them against a known list. This method requires that the 
20 NIDS have direct access to the network traffic to work 
and enough processing power to handle the additional 
workload . 
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SUMMARY OF THE INVENTION 

According to one embodiment of the invention, a 
computerized method for reducing the false alarm rate of 
network intrusion detection systems includes receiving, 
5 from a network intrusion detection sensor, one or more 
data packets associated with an alarm indicative of a 
potential attack on a target host and identifying 
characteristics of the alarm from the data packets. The 
characteristics include at least an attack type and an 

10 operating system fingerprint of the target host. The 
method further includes identifying the operating system 
type from the operating system fingerprint, comparing the 
attack type to the operating system type, and indicating 
whether the target host is vulnerable to the attack based 

15 on the comparison. 

Some embodiments of the invention provide numerous 
technical advantages. Other embodiments may realize 
some, none, or all of these advantages. For example, 
according to one embodiment, the false alarm rate of 

20 network intrusion detection systems ("NIDS" ) is 
substantially reduced or eliminated, which leads to a 
lower requirement of personnel monitoring of NIDS to 
respond to every alarm. This may be facilitated by a 
system in which there is no need to access the network 

25 stream to determine the operating system type of the 
target host. The system may reside anywhere in an 
enterprise and may be used with different types of NIDS, 
even legacy NIDS sensors that do not support passive OS 
fingerprinting. Such a system may free up the NIDS so 

30 that it runs more efficiently and at a faster speed. In 
addition, an offline passive analysis system according to 
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one embodiment facilitates the analysis of target hosts 
that are behind strong or impenetrable firewalls. 

Other advantages may be readily ascertainable by 
those skilled in the art from the following figures, 
5 description, and claims. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

For a more complete understanding of the present 
invention and the advantages thereof, reference is now 
made to the following description taken in conjunction 
5 with the accompanying drawings, wherein like reference 
numbers represent like parts, and which: 

FIGURE 1 is a schematic diagram illustrating a 
system for reducing the false alarm rate of network 
intrusion detection systems by utilizing an offline 
10 passive analysis according to one embodiment of the 
invention; 

FIGURE 2 is a block diagram illustrating various 
functional components a passive analysis tool according 
to the one embodiment of the invention; 

15 FIGURE 3 is a flowchart illustrating a method for 

reducing the false alarm rate of network intrusion 
detection systems according to one embodiment of the 
invention; and 

FIGURE 4 is a flowchart illustrating a method that 

2 0 may be used in conjunction with the method of FIGURE 3 
according to one embodiment of the invention. 
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DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS OF THE INVENTION 

Embodiments of the invention are best understood by 
referring to FIGURES 1 through 4 of the drawings, like 
numerals being used for like and corresponding parts of 
5 the various drawings . 

FIGURE 1 is a schematic diagram illustrating a 
system 100 for reducing the false alarm rate of a network 
intrusion detection system ("NIDS") 108 by utilizing an 
offline passive analysis tool 110 in accordance with one 

10 embodiment of the present invention. In the illustrated 
embodiment, system 100 includes NIDS 108 coupled to a 
link 106 that communicatively couples an unprotected 
network 102 with a protected network 104, a network 120 
that couples NIDS 108 with passive analysis tool 110, a 

15 dynamic host configuration protocol ("DHCP") server 122 
coupled to passive analysis tool 110, and a network 
administrator 112 that utilizes passive analysis tool 
110, as described in more detail below. 

Unprotected network 102 may be any suitable network 

20 external to protected network 104. An example of 

unprotected network 102 is the Internet. Protected 
network 104 may be any suitable network, such as a local 
area network, wide area network, virtual private network, 
or any other suitable network desired to be secure from 

25 unprotected network 102. Link 106 couples unprotected 
network 102 to protected network 104 and may be any 
suitable communications link or channel. In one 

embodiment, communications link 106 is operable to 
transmit data in "packets" between unprotected network 

30 102 and protected network 104; however, communications 
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link 106 may be operable to transmit data in other 
suitable forms. 

NIDS 108 may be any suitable network-based intrusion 
detection system operable to analyze data packets 
5 transmitted over communications link 106 in order to 
detect any potential attacks on protected network 104. 
NIDS 108 may be any suitable combination of hardware, 
firmware, and/or software. Generally, network intrusion 
detection systems include one or more sensors having the 
10 ability to monitor any suitable type of network having 
any suitable data link protocol. In addition, some 
network intrusion detection systems are passive observers 
of network traffic and do not have their own network 
address . 

15 In a particular embodiment of the invention, sensors 

associated with NIDS 108 are operable to examine data 
packets on an IP ("Internet Protocol") network using any 
suitable protocol, such as TCP ("Transmission Controlled 
Protocol") , UDP ("User Datagram Protocol"), and ICMP 

20 ("Internet Controlled Message Protocol"). Upon detection 
of a possible attack on protected network 104, NIDS 108 
is operable to generate an alarm indicating that an 
attack on protected network 104 may have occurred. Alarm 
trigger packets are then transmitted to passive analysis 

25 tool 110 over network 120 along with one or more other 
data packets associated with the alarm for analysis, as 
described in more detail below. 

According to the teachings of one embodiment of the 
present invention, passive analysis tool 110 is a backend 

3 0 application that receives, via network 12 0, one or more 
data packets from NIDS 108 and, using the information 
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associated with the data packets, determines if an attack 
is real or merely a false alarm. These data packets, 
which may be any suitable portion of an information 
stream, include characteristics of the alarm, such as an 
5 attack type and an operating system ("OS") fingerprint 
for the target host so that passive analysis tool 110 may 
analyze the potential attack without having access to the 
network stream on link 106. 

In this manner, passive analysis tool 110 

10 significantly lowers the false alarm rate for network 
intrusion detection systems, such as NIDS 108, in the 
network environment and lowers the requirement of 
personnel, such as network administrator 112, monitoring 
these systems to respond to every alarm. In addition, 

15 passive analysis tool 110 may reside anywhere in an 
enterprise and may be used with different types of NIDS, 
even legacy NIDS that do not support passive OS 
fingerprinting. Passive analysis tool 110 may also, in 
some embodiments, facilitate the analysis of target hosts 

20 that are behind strong or impenetrable firewalls. 

Details of passive analysis tool 110 are described 
in greater detail below in conjunction with FIGURES 2 
through 4. As illustrated in FIGURE 1, passive analysis 
tool is coupled to NIDS 108 via network 120, which may be 

25 any suitable network, or combination of networks, such as 
a local area network, wide area network, global network, 
virtual private network, or any other suitable network. 

Network administrator 112 may be any suitable 
personnel that utilizes passive analysis tool 110 in 

3 0 order to monitor potential attacks on protected network 
104 and respond thereto, if appropriate. Network 
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administrator 112, in one embodiment, has passive 
analysis tool 110 residing on his or her computer in 
order to receive filtered alarms from passive analysis 
tool, as denoted by reference numeral 114. 
5 FIGURE 2 is a block diagram illustrating various 

functional components of passive analysis tool 110 in 
accordance with one embodiment of the present invention. 
The present invention contemplates more, less, or 
different components than those shown in FIGURE 2 . In 

10 the illustrated embodiment, passive analysis tool 110 
includes an alarm input layer 202, an alarm 
interpretation layer 2 04, a target cache look-up 2 06, a 
passive offline fingerprinting mechanism 208, and an 
alarm output layer 210. The general functions of each of 

15 these components are now described before a more detailed 
description of the function of passive analysis tool 110 
is undertaken in conjunction with FIGURES 3 and 4. 

Alarm input layer 202 is generally responsible for 
receiving the data packets from NIDS 108 and determining 

2 0 if the alarm format is valid. If the alarm format is 
invalid, then the alarm is disregarded. If the alarm 
format is valid, then the alarm is sent to alarm 
interpretation layer 204. Alarm input layer 202 is 
preferably designed to be NIDS vendor independent so that 

25 it may accept alarms from multiple NIDS sources 
concurrently with no modification. Alarm input layer 
202, in one embodiment, may also accept alarms from 
legacy NIDS that do not support passive OS 
fingerprinting . 

30 Generally, alarm interpretation layer 204 receives 

the data packets from alarm input layer 2 02 and performs 
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an analysis on the alarm. In one embodiment, alarm 
interpretation layer 204 determines whether the alarm is 
from a supported NIDS vendor. If the alarm is not from a 
supported NIDS vendor, an alert is generated and the 
5 alarm is disregarded. If the alarm is from a supported 
NIDS vendor, then alarm interpretation layer 204 is 
responsible for identifying the attack type, relevant 
operating system type being attacked (e.g., Microsoft 
Windows, Sun Solaris, Linux, UNIX, etc.), the source 

10 address, target network address, the alarm severity, the 
alarm description, and any other suitable parameters 
associated with the alarm. Some of this information is 
used by passive analysis 110 to test if the alarm is real 
or false, as described in more detail below in 

15 conjunction with FIGURES 3 and 4. 

Target cache look-up 206 indicates that a look-up is 
performed by passive analysis tool 110 in order to 
determine if the target host has already been checked for 
the particular attack indicated by the alarm. The look- 

2 0 up may be performed in any suitable storage location, 

such as a local state table or database. 

Passive offline fingerprinting mechanism 208 
performs a passive analysis of the target host by 
identifying, from the received data packets, the 
25 operating system fingerprint of the target host, which 
includes the operating system type, and comparing the 
operating system type to the attack type. An advantage 
of this type of OS fingerprinting is that it requires no 
access to the network stream. Passive offline 

3 0 fingerprinting mechanism 2 08 may store this information 
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in a suitable storage location for later retrieval and 
use . 

Alarm output layer 210 is responsible for taking the 
analyzed data from passive analysis tool 110 and either 
5 escalating or de-escalating the alarm. In other words, 
alarm output layer 210 functions to report a valid alarm; 
i.e., that a particular target host is vulnerable to an 
attack. A valid alarm may be reported in any suitable 
manner, such as a graphical user interface, a log file, 
10 storing in a database, or any other suitable output. In 
one embodiment, a valid alarm is automatically reported 
to network administrator 112 via any suitable method. 

Additional description of the details of the 
functions of passive analysis tool 110, according to one 
15 embodiment of the invention, are described below in 
conjunction with FIGURES 3 and 4. 

FIGURE 3 is a flow chart illustrating an example 
method for reducing the false alarm rate of network 
intrusion detection systems according to one embodiment 

2 0 of the present invention. The example method begins at 

step 3 00 where one or more data packets associated with 
an alarm is received from NIDS 108 by passive analysis 
tool 110. As discussed above, these data packets may be 
any suitable portion of an information stream and may be 
25 communicated to passive analysis tool 110 via network 120 
or other suitable communication means. From the data 
packets, passive analysis tool 110 identifies the attack 
type, as denoted by step 3 02, and an operating system 
fingerprint of the target host, as denoted by step 304. 

3 0 The operating system type of the target host may be 
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identified by passive analysis tool 110 from the OS 
fingerprint, as denoted by step 3 06. 

The attack type and the operating system type of the 
target host are compared at step 308 by passive analysis 
5 tool 110. At decisional step 310, it is determined 
whether the operating system type of the target host 
matches the attack type. If there is a match, then a 
confirmed alarm is reported by step 312. In one 

embodiment, the confirmed alarm is automatically reported 

10 to network administrator 112 in any suitable manner. If 
there is no match, then a false alarm is indicated, as 
denoted by step 314. For example, if the attack type is 
for a Windows system and the operating system fingerprint 
shows a Windows host, then the alarm is confirmed. 

15 However, if the attack type is for a Windows system and 
the operating system fingerprint shows a UNIX host, then 
this indicates a false alarm. This ends the example 
method outlined in FIGURE 3. 

Although the method outlined in FIGURE 3 is 

20 described with reference to passive analysis tool 110 
comparing an operating system type with an attack type, 
other suitable characteristics of the operating system 
may be compared to relevant characteristics of the attack 
type in order to determine if the alarm is real or false. 

2 5 This depends on the type of information passed from NIDS 
108 via the data packets . 

Thus, passive analysis tool 110 is intelligent 
filtering technology that screens out potential false 
alarms while not requiring access to protected network 

30 104. Alarm inputs are received from a deployed NIDS, 
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such as NIDS 108 , and analyzed to determine if an attack 
is real or a false alarm. 

FIGURE 4 is a flowchart illustrating an example 
method that may be used in conjunction with the example 
5 method outlined in FIGURE 3 in accordance with an 
embodiment of the present invention. The example method 
in FIGURE 4 begins at step 400 where DHCP server 122 
(FIGURE 1) is monitored by passive analysis tool 110. 
The present invention contemplates any suitable dynamic 

10 configuration protocol server being monitored by passive 
analysis tool 110. At step 402, lease activity is 
detected by passive analysis tool 110. At decisional 
step 404 it is determined whether a lease issue is 
detected or a lease expire is detected. 

15 If a lease expire is detected by passive analysis 

tool 110, then the system cache is accessed, as denoted 
by step 406. At decisional step 408, it is determined 
whether the target address associated with the lease 
expire is found in the system cache. If the target 

2 0 address is found in the system cache, then the entry is 
purged, at step 410, from the system cache. Passive 
analysis tool 110 then continues to monitor the DHCP 
server. If a target address is not found in the system 
cache, then the lease expire is disregarded, as denoted 

25 by step 412. Passive analysis tool 110 continues to 
monitor the DHCP server. 

Referring back to decisional step 404, if a lease 
issue has been detected, then the system cache is 
accessed, as denoted by step 414. At decisional step 

30 416, it is determined whether the target address 
associated with the lease issue is found in the system 



Attorney Docket No. PATENT APPLICATION 

062891 .1166 

14 

cache. If the target address is found, then the entry is 
purged, at step 418. If the target address is not found 
in the system cache, then passive analysis tool 110 
continues to monitor the DHCP server. 

The method outlined in FIGURE 4 address the dynamic 
addition, subtraction, or modifying of hosts in protected 
network 104 in order that prior knowledge of protected 
network 104 is not required. This saves considerable 
time and money and is more accurate than prior systems in 
which prior knowledge of the network is required. 
Passive analysis tool 110 may more accurately keep track 
of changes regarding the target hosts of protected 
network 104 . 

Although the present invention is described with 
several embodiments, a myriad of changes, variations, 
alterations, transformations, and modifications may be 
suggested to one skilled in the art, and it is intended 
that the present invention encompass such changes, 
variations, alterations, transformations, and 

modifications as they fall within the scope of the 
appended claims. 



